Logarithmic Regret for Episodic Continuous-Time Linear-Quadratic Reinforcement Learning Over a Finite-Time Horizon

We study finite-time horizon continuous-time linear-quadratic reinforcement learning problems in an episodic setting, where both the state and control coefficients are unknown to controller. first propose a least-squares algorithm based on observations controls, establish logarithmic regret bound of order $O((\ln M)(\ln\ln M))$, with $M$ being number episodes. The analysis consists two parts: perturbation analysis, which exploits regularity robustness associated Riccati differential equation; parameter estimation error, relies sub-exponential properties estimators. further practically implementable discrete-time piecewise constant achieves similar additional term depending explicitly time stepsizes used algorithm.